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Abstract 

With support found in the literature for the utilization of push-in, or integrated therapy when 
providing speech language pathology, the use of a set of criteria for determining how therapy 
would be provided was evaluated in a preschool for children with special needs. Using a 5 item 
Likert scale, teachers and speech pathologists were surveyed regarding the criteria’s aide in 
determining how a student should be provided speech therapy. The collaboration that resulted 
from use of the criteria along with an assessment of the progress made by students receiving 
push-in services was also surveyed. Survey data indicated an affinity for the criteria, its 
facilitation of the collaborative process as well as the beneficial impact of push-in therapy on 
students’ development. 


Executive Summary 

Over the course of six months an evaluation regarding the use of a set of criteria used to 
determine whether a student should be provided speech therapy on a push-in or pull-out basis 
was conducted at an inclusive preschool serving children with special needs. Secondary 
components of the evaluation included determining the ability of the criteria assessment process 
to facilitate collaboration between staff as well as the beneficial effects of push-in therapy on 
students’ development. The evaluation was the result of an effort by program administration to 
promote collaboration amongst staff, use of the push-in, or integrated therapy model and finally 
to assist staff in determining how a student should be served. 

The evaluation specifically assessed the opinions of the school’s 23 special education teachers 
and 12 speech language pathologists regarding use of recently designed criteria as an aide in 
determining the contextual provision of speech therapy. The evaluation also assessed the level of 
collaboration that resulted from use of the criteria. The resulting report also included a review of 
the literature pertinent to collaboration and push-in therapy in a special education environment. 

The survey’s results generated data relevant to the survey’s five questions and the two 
participating professional disciplines. Using the SPSS software program, the evaluation found 
that the responding participants overwhelmingly agreed to the following: 

• Use of the push-in/pull-out criteria was helpful when it came to detennining whether a 
student should be provided speech therapy services in isolation or in the presence of other 
students in the contextual setting of the classroom. 
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• The process of having the special educator and the speech therapist use the push-in/pull¬ 
out criteria aided in the collaborative effort between the two professionals. 

• The provision of push-in speech therapy was beneficial to the receiving student. 

A majority of the participants provided commentary and expressed support for their 
interdisciplinary colleagues as well as the belief that most students showed progress from push- 
in therapy, especially when it came to pragmatic skill development. 

While the evaluation’s results were encouraging, a more thorough examination of student 
progress be considered in order to provide enhanced justification for the push-in model. 

Introduction 


What is Push-in Therapy and Does it Help Collaboration ?: Collaboration and the push-in 
model. This report is an evaluation of integrated, or push - in therapy at an inclusionary early 
childhood program serving children ages 3-5 with special needs. The process evaluated involved 
assessing several factors relating to integrated therapy including the usefulness and effectiveness 
of a criterion based instrument used to determine whether a student should be provided therapy 
in an integrated manner. In addition, the evaluation will determined whether collaboration 
between speech-language pathologists and special education teachers was enhanced as a result of 
this process. Finally, the evaluation attempted to detennine if students benefited from integrated 
therapy. 

Collaboration amongst professionals in the special education environment is considered best 
practice and viewed as an opportunity to enhance the development of skills and abilities of 
students with special needs (McWilliam & Young, 1996; Barnes & Turner, 2001). Collaboration 
itself is defined in a myriad of ways. Friend & Cook (1992) state collaboration is “a style for 
direct interaction between at least two co-equal partners voluntarily engaged in shared decision 
making as they work toward a common goal” (p.5). Others, including Rainforth and England 
(1997), Wade, Welch, and Jensen, (1994), and Welch (1998b) make reference to the cultural and 
contextual setting, the need for respect for each participant and a sense of problem ownership by 
each team member. 

Collaboration’s ability to enhance the educational and therapeutic intervention 
provided to students with special needs is further enhanced when alternate modes of 
therapeutic interventions, such as push-in therapy, are provided to students in the classroom 
setting (Bames & Turner, 2001; Ritzman, Sanger, & Coufal, 2006). Dule, Korner, Williams and 
Carter (1999) add that “integrated therapy” (p. 244) has been found to aid in collaborative 
approaches that bring professionals together to help create quality educational programs with 
high levels of student involvement. McWilliam (1996) adds that “until conclusive evidence is 
found to support pull-out therapy that involves minimal contact with classroom teachers, 
integrated therapy is more compatible with current philosophical trends in early intervention” (p. 
101 ). 

Push-in, or integrated therapy, is the provision of therapeutic intervention in the context of the 
classroom setting (Cross, Traub, Hutter-Pishgahi, & Shelton, 2004). This is in contrast to the 
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more traditional model of intervention, commonly called pull-out therapy, where the therapist 
removes the child from the classroom and provides intervention in an isolated setting absent of 
other children (Harn, Bradshaw, & Ogletree, 1999). 

At the preschool where the program evaluation is occurred, a high degree of collaboration had 
already been incorporated into its service provision. Collaboration at the school had developed to 
the extent that it has become an integral aspect of the organization’s culture and embodies what 
Tulbert (2000) calls a collaborative ethic. The collaborative ethic embodies the social, cultural 
and structural constructs of collaboration as exemplified by shared values and actions that 
support and encourage the collaborative process while also respecting one another’s discipline 
specific skills and role in the process. Even with this high degree of collaboration and a sense of 
a collaborative ethic, school administration, including the author, believed the school was ready 
to emphasize the use of integrated or push-in therapy as a component of the collaborative 
approach to educational intervention. For this reason, program administration had undertaken an 
effort o promote the use of integrated/push-in therapy as one alternative in a continuum of 
options (Ritzman, et. ah, 2006) available to therapists and other service providers. 

The setting for this evaluation is a state approved and publicly supported early childhood special 
education preschool with an enrollment of slightly over three hundred children. The participants 
in the evaluation will include up to twenty three special educators and up to twelve speech 
language pathologists. 

Students who attend the preschool are referred to the school by the county in which they reside 
or their school district. Those students below age three are referred through the Early 
Intervention (E.I.) Program while those who are between 3-5 years of age are served through the 
Preschool Special Education Program. Services for all students are determined by either an 
Individualized Family Service Plan (IFSP) or an Individualized Education Plan (IEP). The 
former is for students in E.I. while the latter is for students of preschool age. Both IFSPs and 
IEPs specify what special education and related services students are to receive and at what 
frequency. Variations in service and frequencies are in response to the student’s level of 
developmental delay or disability and are detennined by committees of individuals appointed by 
the municipality or local school district (Friend, 2007). Service providers, such as this preschool, 
are not involved in determining service levels. 

Program Description'. A philosophical and theoretical history of therapy provision in a 
collaborative special education environment. The struggle, past and present. 

Historically, interaction amongst special education service providers has been a contentious issue 
filled with debate and discourse (Palfrey, Singer, Raphael, & Walker, 1990; Tourse, Mooney, 
Kline & Davoren, 2005). On the one hand, many of the current related services, such as speech 
language pathology, traditionally dictated a medically oriented approach to correcting 
dysfunction that frequently required taking students out of their classroom to “receive services in 
a separate therapy resource room” (ASHA 2000, p. 5). This kind of separate and sometimes 
fragmented service was commonplace and difficult to alter (Harn, et. al, 1999). 
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As noted by Rainforth and England (1997), not every special education provider, or for that 
matter related service provider, was enamored with the idea of collaboration. Some in the special 
education community perceive collaboration as counter to professionalism and are content with 
old traditions, the maintenance of teachers autonomy and the “cellular structure” (Lortie, 1975, 
p. 149) of the classroom. Some, including York, Rainforth, and Giangreco (1990), as cited by 
Welch (1998a) expressed concern with the hype associated with collaboration, believing it has 
been viewed as “one of the many bandwagons in the parade of education reform rhetoric” (p. 

26). 

Even after decades of “policy and practice guidelines encouraging an integrated and 
comprehensive approach to service delivery” (Whitmire & Eger, 2004, p. 27), current special 
education practice is rife with conflicting intent and expectations. Weintraub and Kovshi (2004) 
document the continued reliance by occupational therapists on the traditional pull-out form of 
service provision, with that model preferred by over two thirds of those surveyed. Others, such as 
Dule, et. al. (1999), McWilliam and Young (1996), Kaminker, Chiarello, O’Neil, and Dichter 
(2004), and Ritzman, et. al., (2006) all document the past and continued preference for isolated 
service provision, even in the face of evidence noting the benefits of more integrated, push-in 
models (ASHA, 1991). Even within the world of education, there have been difficulties faced by 
special educators who want to collaborate with general educators in an effort to assist in the 
successful inclusion of children with special needs into the general education classroom 
(Laycock & Gable, 1991). And although current practices such as inclusion has lead to 
dissolution of the cultural divide between general and special education, there remains a host of 
issues that constrain the ability of professionals to collaborate effectively (Bruder & Dunst, 2005; 
Ham, et. al., 1999; Friend, 2000; Rainforth & England, 1997; Welch, 1998b). 

The difficulty experienced by many when it comes to the integration of services within the team 
structure (Downing & Bailey, 1990) relates in many ways to the fact that, as noted by Friend 
(2000), collaboration is hard work, takes time and requires skillful execution. But even before 
collaboration can be successfully implemented, the process requires prospective team members 
to exchange knowledge and insight into each other’s professional storehouse of expertise 
(Rainforth & England, 1997). Within the integrated team environment, participants are likely to 
engage in a collaborative setting that emphasizes communication and cross discipline 
intervention strategies (Carpenter, King-Sears, & Keys, 1998; Downing & Bailey, 1990; Stepans, 
Thompson, & Buchanan, 2002). Even those who are supportive of the collaborative and 
integrated service processes can find themselves overwhelmed. Just the need for regular 
communication can be a challenge and burden. As cited by ASHA (2006), lack of time for 
planning, collaboration, and meeting with teachers has been cited as the second greatest 
challenge to effective practice after caseload size. In particular, 66 - 81% of speech therapists 
saw the lack of time for collaboration as a serious issue. These hindrances point to the need for 
education and training (Welch, 1998b) as well as administrative support (Moore-Brown, 1991) 
and flexible scheduling (ASHA, 1999) in the continued development of collaborative and 
integrated service delivery systems. 

These practices, obstacles and concerns notwithstanding, whether we are talking about services 
for infants and toddlers in early intervention (Bruder & Dunst, 2005), school-age children placed 
in inclusionary settings (Carpenter, et, al., 1998) or students with multiple disabilities (Campbell, 
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1987, Downing & Bailey, 1990), the idea of having professionals collaboratively providing 
services and creating common goals (Dule, et. ah, 1999) has become accepted by many in the 
field. Other studies have shown that collaboration can benefit students, teachers and others in a 
variety of ways, such as the simple sharing of resources and expertise across discipline 
boundaries (Wade, et. ah, 1994). In general, collaboration is now viewed as a powerful tool for 
helping teachers serve students with disabilities (Brownell, Adams, Sindelar, Waldron & 
Vanhover, 2006) while push-in therapy, or integrated services, has grown in importance in the 
field (Wilcox & Shannon, 1996). 

While collaboration, and the integration of therapeutic services into the classroom setting, is 
relevant for most, if not all related services, it is much more applicable to the provision of speech 
therapy. One of the primary reasons for this is the prolific application of speech therapy in a 
special education environment. This is certainly true of the student population at the preschool in 
question. Of the total enrollment of 200 who were students at the time in the center-based 
component of preschool, 168 receive speech therapy. This is consistent with state-wide data that 
show an average of 88% of all preschoolers with special needs that attend center-based classes 
receive speech therapy. This is in contrast with other therapies, all of which are provided much 
less often, such as Occupational Therapy (51%) and Physical Therapy (25.5 %) (MGT of 
America, 2007). 

Program history -philosophical and theoretical foundations ofpush-in sendees Even as early as 
the 1970s, some in the field, including speech-language pathologists, recognized the benefits of 
learning therapeutic skills in the natural environment (Wilcox & Shannon, 1996; Elksnin & 
Capilouto, 1994). Later, Barnes and Turner (2001) documented the benefits of joint intervention 
as carried out by special educators and occupational therapists, while Rapport and Williamson 
(2004) illustrated the need for collaboration by physical therapists. Many point to the enhanced 
generalization that occurs in learning when the intervention is provided in an integrated fashion 
(Warren & Horn, 1996). These same authors emphasize this point with the following six 
principles of integrated therapy (p. 121): 

1. Therapy and instruction should occur in the child’s classroom. 

2. Other children should be present. 

3. Therapy and instruction should be embedded in ongoing classroom routines and 

1. activities. 

4. Therapy and instruction should follow the child’s attentional lead. 

5. Goals should be functional and immediately useful. 

6. The primary role of the therapist is as a collaborator with other members of the child’s 

team. 

In McWilliam (1995), the extent to which therapy is provided in an integrated setting can vary 
according to several dimensions related to the above principles including; the physical location 
and setup of the treatment area, what other adults and children are in attendance at the time, how 
the therapist goes about providing service, the goals to be addressed, and the specific activities 
incorporated into the session. When it comes to speech-language therapy, speech therapists have 
identified “the need to integrate communication and language goals with other educational goals 
to achieve academic and social success” (Wilcox & Shannon, 1996, p. 218). 
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Along with the philosophical underpinnings of the collaborative and integrated therapy approach, 
there are also legal mandates to work collaboratively and integrate services. This has been the 
case at the federal level since the passage of the first federal laws requiring the provision of a 
free and appropriate public education beginning in 1975 with the passage of P.L. 94-142, the 
Education for All Handicapped Children Act (Weintraub & Kovishi, 2004). The reauthorization 
of Individuals with Disabilities Education Act (IDEA) in 1997 extended special education 
services and the need for collaboration to infants and toddlers ages birth to three after earlier 
reauthorizations applied the mandate to preschoolers (Bruder & Dunst, 2005). These landmark 
pieces of legislation also included a continuum for the delivery of service, recognizing that a 
range of opportunities is necessary to meet the needs of all students with special needs (Friend, 
2007). 

Besides the legislative mandates noted above, several professional organizations have 
incorporated into their member standards various measures for collaboration and integrated 
services (Welch, 1998a). The Council for Exceptional Children (CEC) Special Education 
Content Standard #10 notes that “special educators routinely and effectively collaborate with 
families, other educators, related service providers, and personnel from community agencies in 
culturally responsive ways” (Friend, 2007, p.515). The Interstate New Teacher Assessment and 
Support Consortium (INTASC) also speak of the relationships that teachers foster in support of 
their students’ learning (Friend, 2007). In addition to the influence of both legislative and 
professional standards and expectations, much of the change in attitudes towards collaboration 
and integrated services is as a result of societal and cultural influences (ASHA, 2000). 

Along with the growing ethnic and cultural diversity taking place in American society, the 
growth of the inclusion movement and its emphasis on placing students with special needs in 
typical classrooms has placed greater pressure on teachers and therapists to work together and 
share service strategies. This has resulted in therapists of all disciplines providing service in 
more eclectic settings, not just the speech therapy treatment room (ASHA 2000). Finally, current 
practice has come to resolve the disputes of the past by emphasizing the need to maintain a 
“student-centered focus” (ASHA, 2000, p.2) when supporting learning while recognizing the 
classroom as the “most fertile ground” (Wilcox & Shannon, 1996, p. 221) for skill generalization 
and the “anchor” (Wilcox & Shannon, 1996., p. 222) in the provision of integrated speech- 
language therapy. 

Evaluation Setting, Background and Current Practice 

The history of collaboration and integrated services was in many ways replicated in the setting 
for this evaluation. Collaboration at the preschool’s various service providers and the integration 
of related services into the classroom setting had been filled with some barriers, a few bumps and 
most recently a fair amount of success. Upon my arrival as the school administrator in June 
2002, the concept of collaboration was rather fractured if non-existent within the organization. 
The culture of the school was in many ways contrary to the concept of teamwork, with the 
presence of a fair amount of individualism along with a few, small cliques at work within the 
teacher ranks. Meanwhile, the related service departments were entities unto themselves, seeing 
themselves as separate, distinct departments with limited connection to the overall organization. 
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Team meetings amongst the individual service providers were essentially non-existent and 
almost all therapy was provided in a pull-out manner. Many of the therapists, including one 
senior therapist with over fifteen years of experience, had never provided therapy in a setting 
other than an isolated treatment room. 

The lack of a collaborative culture at the preschool was in stark contrast to the author’s 
experience and training. Besides the accolades afforded to collaboration and integrated therapy 
as exemplified in the literature discussed here, the author had experience with the benefits of 
these best practices while serving in administrative positions at another organization. 

The adoption of a collaborative approach with integrated therapy at the author’s former school 
was a process similar to the one experienced at the preschool in question. 

After a short period of time assessing the culture of the organization and identifying some of the 
key stakeholders, the author began to discuss the concept of collaboration, teamwork and 
integrated service delivery. Some of the author’s colleagues were encouraged and excited by the 
concepts, while others expressed trepidation or outright opposition to the thought of having 
“their” way of working altered or in some way impacted by the new administrator. 

Recognizing that there was a mix of support and opposition, the author felt it best to introduce 
collaboration in a measured, calculated fashion. In conjunction with a few key members of the 
staff, including other administrators and select teachers and therapists, a “Collaboration Synergy 
Committee” was established. The committee was charged with the goal of creating a 
collaborative culture within the school through staff training and the institutionalization of 
practices and procedures that would result in the coming together of staff to synergize the 
collaborative movement amongst service providers. 

While synergy was the ultimate goal, the committee first felt the need to provide education and 
guidance to the staff and therefore distributed literature highlighting best practice and trained the 
staff in some of the essential facets of collaboration, effective team work as well as integrated 
therapy. We also felt it necessary to apply some of the principles being learned and so mandated 
the establishment of regularly scheduled team meetings. A team meeting calendar was first 
distributed in September 2002. At the same time the author recognized the need to take measured 
steps and not give staff the impression that collaboration was being forced upon them. For that 
reason the author was content with simply seeing the successful implementation of team 
meetings as the goal for the year. 

As the school year progressed, small accomplishments were being made in the evolution towards 
collaboration. Team meetings were occurring on a regular basis and producing positive feedback 
from the staff. There were some though who were not so enamored with these new practices. 
Many staff, teachers and therapists, felt the meetings took away from their time to do paperwork 
and didn’t produce worthwhile results. Other staff gave the sense that they now needed to justify 
their methods and practices to their colleagues. The committee attempted to address these 
concerns by helping with time and agenda management. Nevertheless, the general assumption 
was that most committee members believed the exercise in team meetings was an overall 
success. Today team meetings have become an institution of the school as exemplified by the 
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team meeting data from October 2007. An analysis of this data showed 21 out of 23 classroom 
teams had at minimum the mandated two team meetings during the month. In addition, at least 4 
out of 4 team members attended 38 out of the 50 meetings (76%) that were held during the 
month while another 10 meetings had 3 members present, resulting in 96% of all meetings 
having at least three members in attendance. 

The move to a more highly evolved form of collaborative therapy had been a slow process. 
Managing a school is challenging, especially one that has a workforce with diverse skill sets and 
professional allegiances. Whether directly or indirectly, many of these issues impacted on the 
momentum and motivation needed to affect change at the preschool. Besides some continuing 
staff resistance to the collaborative effort as well as the simple lack of staff to fill necessary 
positions, other issues such as budget constraints or governmental audits came to occupy my 
time and the time of others. For this reason, the evolutionary process seemed to slow to an 
almost imperceptible crawl during most of the 2005-06 school year. This was especially true 
with regard to integrated, push-in therapy. 

The opposition at the preschool by some to the push-in model in particular and to collaboration 
in general is, as noted earlier, consistent with the issues seen in the literature. In addition, there 
were several other misunderstandings prevalent in the school and evident in the educational 
community relevant to collaboration, integrated therapy and the intent of both. First, to 
coordinate or cooperative does not equate to collaboration. Coordinating is a managerial process 
designed to accomplish certain tasks while cooperation is a process where people may agree to 
certain activities, which may or may not be truly beneficial to all involved parties (Welch, 

1998b). 

Friend (2000) highlights a few added realities; including the misunderstanding that collaboration 
and its application of integrated therapy comes naturally. It is not uncommon to speak to 
professionals and hear many of them comment on how difficult collaborating is. One must also 
not assume that everyone is collaborating or that more collaboration is necessarily better. Besides 
the skill collaboration requires, it takes time and effort to implement and maintain. With the 
workload already beyond what many would consider appropriate for teachers and others, adding 
a slew of meetings or other tasks to their schedule is sure to elicit exasperated pleas for relief. 
Even if one is committed to collaboration, adding more time to the process does not necessarily 
translate into better outcomes. Dule, et. al. (1999) identified the “ambivalence” (p. 259) and 
“significant difficulties” (p. 260) therapists experience when involved in a collaborative team 
approach. In fact, additional time in the collaborative process could result in less time for 
engagement in the actual work needed to accomplish the mutually agreed upon goals. 

Finally, collaboration is not always a feel-good exercise. In many instances, collaboration 
produces conflict and requires resolution in order for the group to move forward (Friend, 2000). 
Conflict can be very difficult to overcome, and this has been the case for us at the Marcus 
Avenue School. Collaboration is not about liking someone or being liked, it is about trust, 
respect and outcomes. Collaboration is not a standalone process being employed for its own 
sake. It is a technique designed to accomplish a goal in a manner not attainable working apart as 
individuals (ASHA, 1991). 
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Ingredients necessary for successful collaboration and integrated therapy. 

Along with the need for trust, respect and commitment to a process, collaboration and its product 
of integrated therapy requires a sense of professional efficacy and competency in overcoming 
any hesitancy associated with collaboration and the “joint-ownership and responsibility” (ASHA, 
1991, p. 2) that comes with the process. Spann-Hite, Picklesimer, & Hamilton (1999) note the 
willingness of teachers to allow speech language pathologists to participate in classroom 
activities was highly dependent on their own sense of efficacy as teachers and their ability to 
manage students with behavioral disabilities. Trimble and Peterson (1999) note the enhanced 
sense of efficacy teachers felt when administrators lessened the need for the submission of lesson 
plans. The study also showed heightened collaborative effectiveness by team members along 
with improved student outcomes as a result of the administrative support associated with the 
collaborative undertaking. Weintraub and Kovshi (2004) reiterate these findings with their own 
study of occupational therapists. In this study the authors found a correlation between the 
occupational therapists’ level of interest in collaboration and integrated therapy and their own 
level of professional confidence. The more confident the therapists were with their own 
competency, the more likely they were to collaborate, provide service in the classroom setting 
and in the presence of the classroom teacher and others. 

Along with the attitudes, beliefs and values that individuals may have towards collaboration, 
organizations as a whole and groups within larger organizations may substantially influence 
collaborative development through several variables, including the openness versus 
restrictiveness of the relationships within the group (Wade, et al., 1994). Groups with restrictive 
or closed relationships, such as the culture found at the preschool when I first arrived in 2002, 
are environments with barriers that prevent optimal functioning of the group. An unproductive 
relationship can therefore produce far reaching and long term consequences, for a therapist 
employing a pull-out model of therapy will have less and less contact with the other team 
members and therefore less of an opportunity to develop a collaborative relationship (Weintraub 
& Kovshi, 2004 

As noted above, there were many at the preschool that were completely unfamiliar with the 
concept or practice of push-in therapy, and so the prospects of altering this behavior looked poor. 
Discussions continued though and eventually we were able to institute a group therapy program 
that seemed to garner praise and acceptance from the participating teacher and therapists. This 
activity entailed the collaborative engagement of all members of a classroom team in the 
provision of therapy to a group of students within one class. While the intent was well-meaning, 
there was much concern with the process and its appropriateness in addressing individual student 
needs. In particular, some staff were concerned the group sessions didn’t allow them to work 
effectively with their students since they had to divide their time amongst the other students in 
the class. This required review and was on top of the committee’s agenda. 

With a stable cadre of staff and the relative absence of other major issues, the committee was 
finally able to attend to the resumption of the collaborative therapy process with a renewed vigor 
in the summer of 2007. With the involvement of few new senior therapists and others who were 
conducive to push-in therapy, the collaborative committee moved forward to address the issues 
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left on the table over a year earlier, including an effort at justifying and documenting why some 
students received therapy on an integrated, push-in basis whereas others did not. 

Program Characteristics. The question as to why some students were provided therapy on a 
push-in basis and others were not spurred the discussion that, for many staff, had always been at 
the heart of the matter. In essence, many staff simply did not know how to provide push-in 
therapy and where asking for guidance. Collectively, the committee decided to create a set of 
criteria and written directives for push-in therapy. 

In an effort to gain broad-based support, the committee believed it was necessary to expressly 
involve as many of the staff as possible in the criteria creation process. An invitation to 
participate was extended to senior clinicians of each department or all members of a department 
that lacked a senior clinician. Over the summer the process resulted in the creation of criteria for 
the provision of push-in and pull-out speech therapy. Creation of the criteria also resulted in the 
formulation of checklists (Attachments A & B) for use when assessing the feasibility of 
providing therapy on a push-in or pull-out basis. It is this criteria, its associated checklists and 
what happened when they were implemented that is the subject of the program evaluation. 

By late fall of the school year the Push-in/Pull-out Criteria Program was implemented across the 
school with the classroom’s speech-language pathologist serving as the team’s facilitator and 
task master. The criteria and checklists were implemented and utilized in the assessment of 
children potentially benefiting from push-in therapy. Giving the speech-language department the 
lead in this endeavor is in many ways a logical choice since speech-language pathology had 
applied an “increasing emphasis on the importance of and need for provisions of speech and 
language services from a perspective that incorporates integrated practices” (Wilcox & Shannon, 
p. 217) while also recognizing the classroom as a “natural and appropriate context for the 
facilitation of communication and language skills” (Wilcox & Shannon, p. 237). 

Program Clientele. The program where the evaluation took place exists in a multi-service 
organization with a total enrollment of over 350 children. The component of the school that is 
the focus of this evaluation was the center-based division of the school that has an enrollment 
over 200 children age 2-5 with a wide variety of developmental delays or disabilities. These 
students are authorized to attend one of the school’s 23 classrooms on a full-time basis. Note that 
there are approximately 150 other students who receive services provided by the school but for 
various reasons were not included in the push-in assessment process. Twenty-five of these 
students were not authorized to receive classroom instruction and only received limited 
therapeutic service at home. One-hundred and twenty-five other children were not in need of 
special education intervention at all and were not eligible to receive therapy of any kind. 

All of the students with special needs were deemed eligible for special education preschool 
services under federal law and state regulation. The delays or disabilities presented in the 
students ranged from those with severe, multiple disabilities with medical fragility in need of 
intensive therapeutic intervention to those with relatively mild involvement who functioned at 
levels close to that of a typically developing child. Of the 200 children eligible to receive special 
education services, 168 were authorized to receive speech-language pathology as a component of 
their individualized education plan. Services were authorized due to delays or disabilities related 
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to communication, articulation, pragmatics, or oral motor or swallowing issues. As a result of 
these issues, the identified students were mandated on their I.E.P. to receive a total of 450 thirty- 
minute therapy sessions per week. 

The 168 students cited above who received speech-language therapy, their teachers and 
therapists, were the participants in the evaluation. A total of 12 speech-language pathologists 
provided the 450 sessions to the 168 students cited above. This translates to an average of 2.678 
sessions received per student per week and 37.5 sessions provided per therapist per week or 7.5 
sessions per day. A full day’s caseload is set at 10 sessions per day for a therapist working a full 
seven hour day. 

Of the twelve speech-language pathologists, three were required to manage a part-time case-load 
while the remaining nine individuals had a full-time caseload. These nine full-time pathologists 
saw 148 students for 399 sessions or an average of 44.33 sessions per week. The part-time 
caseload therapists who served the remaining 20 students had 51 mandated sessions. Of them, 
one therapist saw only 1 student for 3 sessions per week while the other two therapists served 6 
and 13 students and provided 14 and 34 sessions, respectively. 

Funding Sources. The push-in process is itself unfunded but is financially supported as a service 
provided by the school. The school was funded through the state government and local 
government via the payment of student tuition as established by the New York State Department 
of Education under the provision of a free and appropriate public education as mandated by 
IDEA 2004 (Friend, 2007). Annual tuition charges paid by a combination of county and state 
agencies average forty thousand dollars per student with variations depending on whether the 
child is approved to attend a six week summer session. All students attend as day students. The 
program does not operate a residential component. Total operating budget approximated ten 
million dollars. 

Program Costs. Actual costs to the push-in assessment process were negligible (miscellaneous 
copying, etc.) but the school itself incurred significant costs relating to personnel and other 
operating expenses. As a non-profit organization, the program was compelled to stay within 
budget while ensuring the school provides services as per the needs of the students. 

Administrative Structure. The school’s structure is consistent with other non-profit social service 
and educational organizations and mirrors to a certain extent schools and school districts. The 
author of this evaluation was the program administrator. The author reported to an associate 
executive director and has several supervisors and senior clinical and educational staff reporting 
to him. The school employs over one hundred and sixty full time staff including program 
administration, educational faculty, clinical therapists, paraprofessionals, administrative support 
staff and fiscal personnel. 

Contextual Setting 

Program Location. The preschool in question is located in a suburban county outside of New 
York City. Besides its service to infants, toddlers and typical preschool age children, the program 
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served a wide variety of preschoolers with special needs from the suburban county it is located in 
as well as one of the boroughs of New York City. 

Political Climate. The political climate of the school was typical of many organizations with 
multiple stakeholders, ranging from parents, regulators, senior management and those with 
power and influence. The program had its own political overtones with much of it stemming 
from the interpersonal barriers to collaboration cited in the above Program Description. 

Socio-demographic profiles. The preschool served approximately two-hundred preschool age 
children with special needs, inclusive of 168 children that received speech-language pathology as 
a component of their IEP. Children were enrolled as full time students, attending on a typical 
school calendar from 8:30 am -2:30 pm, Monday - Friday. Besides the usual ten month school 
program, most students also attend a six week summer session. As noted above, the student 
catchment areas included the suburban county outside of New York City in which the preschool 
is located in as well as one of the outer boroughs of New York City. The preschool was 
culturally and economically diverse and represented the socio-economic and demographic 
diversity of these regions. The most noticeable demographic difference with the student body 
was that almost 70 % of the students with special needs (141) were boys. This is consistent with 
national data showing 67 -73% of students with special needs are boys (MGT of America, 2007). 


Evaluation Purpose and Questions 

Under the premise that the team approach offers the opportunity to deliver quality services to 
students with intense need for assistance (Dule, et. ah, 1999), the effort evaluated the assessment 
process used to determine if students would be provided speech therapy through a push-in or 
pull-out model. It also evaluated the impact the assessment process had on the overall level of 
collaboration amongst team members. 

This formative program assessment gathered data through a survey from twelve speech language 
pathologists and twenty three special educators who implemented a set of criteria on how 
students will be provided speech therapy, either in the classroom as a push-in session or out of 
the classroom as a pull-out session. 

The purpose of this evaluation (and the relevant questions) was to: 

1. Evaluate both individually and collectively, the staffs perceptions of the usefulness and 
effectiveness of the push-in/pull-out assessment instrument in choosing students for push-in 
speech therapy. 

Q: Was the push-in/pull-out assessment instrument useful and effective in determining which 
students should be seen via a push-in or pull-out model? 

2. Evaluate whether the teachers and speech therapists viewed use of the push-in/pull-out 
assessment instrument as an enhancement to collaboration. 
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Q: Did the process of using the push-in/pull-out assessment instrument result in enhanced 
collaboration between teachers and speech therapists? 

3. Evaluate whether the staff, again individually and collectively, perceived the actual provision 
of push-in therapy resulted in an enhanced state of collaboration between them. 

Q: Did the provision of push-in therapy enhance collaboration between teachers and therapists 
over the course of the evaluation period? 

4. Evaluate whether the staff perceived that the implementation of push-in therapy was 
successfully incorporated into the classroom. 

Q: Was push-in therapy successfully incorporated into the classroom setting? 

5. Evaluate whether the staff found push-in therapy beneficial to those students in receipt of the 
service. 

Q: Did teachers and speech therapists determine push-in therapy to be beneficial to students 
over the course of the evaluation period? 


Methodology 

A. This evaluation utilized nonexperimental, causal-comparative research where no manipulation 
of the categorical independent variable occurs and participants are not randomly assigned. It can 
also be viewed separately as a management oriented, consumer oriented and product oriented 
evaluation. The evaluation itself was formative in scope but had summative aspects to it, 
especially as it pertains to the continued use of the push-in/pull-out criteria assessment tool. The 
evaluation relied primarily on quantitative research methods to produce useful data. 

The formative context of the evaluation was applicable to the intent to collect data regarding the 
effectiveness and usefulness of the assessment tool in determining whether students should be 
provided therapy on a push-in or pull-out basis. The formative nature of the evaluation also 
pertained to the staffs ability to make collaborative decisions while engaged in the push-in/pull¬ 
out assessment process. The data will provide administration with infonnation that will be used 
to guide future modification of the assessment tool as well as continued evolution of the push-in 
therapy approach. The summative nature of the evaluation may result if the data indicate that the 
push-in/pull-out assessment tool is ineffective in relation to its intended use. 

The management oriented evaluation approach was also used to assist school administration to 
decide on the modification and/or continuance of the push-in/pull-out assessment process as well 
as what actions the school would take regarding future collaborative efforts. 

Stufflebeam’s CIPP Evaluation Model (Smith & Farr, 1971) was used to collect data regarding 
four aspects of the assessment tool and process: 

1. Context - was the resulting push-in process successful in the classroom context? 

(Evaluation Purpose/Question # 4) 
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2. Input - did the assessment tool (an input) lead to efficient and effective selection of students 
for push-in therapy. (Evaluation Purpose/Question #1 ) 

3. Process - did the assessment process as well as implementation of push-in therapy lead to 
enhanced collaboration between teachers and therapists? (Evaluation Purposes/Questions #2 & 

3) 

4. Product - did push-in therapy benefit those students who received speech-language therapy 
in this manner? (Evaluation Purpose/Question # 5) 

The consumer evaluation approach was used to determine if the push-in/pull-out assessment tool 
and process was viewed by the staff as effective and efficient in detennining whether students 
should be seen on a push-in or pull-out basis as well as whether the tool and/or process enhanced 
collaboration between teachers and therapists. 

B. The push-in/pull-out assessment instrument was introduced by the identified speech therapists 
to their associated special educators beginning in September 2007. Assessments commenced 
immediately thereafter. The teachers and therapists were instructed to assess each student 
mandated for speech therapy using the assessment instruments. 

C. The survey instrument was a five item questionnaire (Attachment C) incorporating a five 
point Likert scale. The instrument will ask respondents to answer the questions from the 
following possible responses: (1) Strongly Disagree, Disagree, Not Sure, Agree and Strongly 
Agree (5). A comment area allowed for respondents to provide additional information. 

The survey was distributed via staff mailboxes and asked to be returned within four weeks. 
Adhering to this timeline allowed therapists and teachers up to six months after distribution of 
the push-in/pull-out assessment instrument to work with students and make determinations as to 
the benefit of the push-in/pull-out assessment instrument and the intended collaborative process. 

Reliability of the survey instrument was limited due to several factors including limited 
robustness of measure (only one measure), limitations of reliability as a result of only one 
measurement and tool being used. Questions regarding the survey’s validity and sensitivity 
pertain to the survey’s ability to sufficiently measure the intent of the questions and this 
evaluation and its sensitivity when it comes to its ability to accurately measure collaboration and 
students improvement as a result of push-in therapy. 

D. Interviews were not conducted for this evaluation. In this setting, anonymous, quantitative 
data gathering is deemed more reliable than qualitative interviews. The evaluator believed staff 
would not express themselves in an honest and forthright manner considering the perceived 
interest and investment the evaluator has in the development of a collaborative approach and 
integrated therapy model within the school. 

E. The identified sample of participants (35) consisted of the entire population (N) of teachers 
and speech therapists employed by the school with a center-based assigmnent. In addition, all 
were involved in the push-in/pull-out determination process. Statistical control was achieved by 
using a case control design since the “targets (special educators and speech therapists) are drawn 
from a specialized population with distinctive clinical characteristics” (Rossi, Lipsey, & 
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Freeman, 2004, p. 279). A binary coding process was used to identify which type of professional 
(speech therapist vs. special educator) had returned the survey. 

Selection of staff participants was non-random using a nonequivalent comparison design (Rossi 
et.al., 2004) while inclusion of students into the push-in/pull-out criteria assessment process used 
a “regression-discontinuity design” (Rossi, et. ah, 2004, p. 288), where participants are selected 
thru collaborative decision making and the application of qualifying condition(s) such as whether 
the students could benefit from push-in therapy. 

The study was cross-sectional and explanatory in nature and the survey based research is 
purposive in nature using a disproportional stratified simple random sample - equal probability 
selection methodology (Johnson & Christensen, 2008). All teachers and SLPs were the intended 
sample (N = 12 + 23 = 35). The actual sample totaled 27 participants (n = 17 + 10 = 27). The 
difference in intended population and eventual sample was attributed to several factors. This 
includes a case of differential participant attrition when one special educator departed for 
maternity leave prior to distribution of the survey as well as the cut-off of survey submission due 
to time constraints. 

F. The data described the perceptions the identified staff has regarding the push-in/pull-out 
assessment instrument, its usefulness and effectiveness, as well as the level of collaboration that 
resulted from use of the instrument. Finally, the data gave insight into the staffs’ perception 
regarding the benefits of the push-in process. 


Results 

Surveys were submitted by 17 special educators and 10 speech therapists, representing 74% and 
85% of each discipline, respectively. The surveys were returned to this researcher through the 
interoffice mail process. The total returned surveys (N = 27) represented a response rate of 77%. 
As noted above, one special educator was unable to participate due to the commencement of 
maternity leave prior to survey distribution. The reason for the other unreturned surveys (7 or 
20%) can be attributed to a number of factors including possible lack of interest (4) and missed 
submission deadline (3). 

As noted in Table 1, the results showed a mean score of 4.0 (“Agree” as per the Likert scale used 
in the survey) for the five survey questions with a standard deviation of only 0.19. The similarity 
in scores amongst the respondents can be taken as an indicator of consistency in thinking 
between the two disciplines and barometric of their collaborative relationship. 

Table 1. Mean Aggregate Scores for Questions #1-5 


Q.# 

Mean 

Scores 

Qi 

4.0370 

Q2 

3.8889 

Q3 

4.0370 
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\$4~ 

4.0741 

Q5 

3.8889 


In addition to the consistency in mean scores, there was a strong correlation in the answers to 
questions 1 and 2 (questions regarding the Push-in/Pull-out Criteria and Criteria Tool) and 
questions 3 and 4 (questions regarding collaboration and push-in therapy) as illustrated in Tables 
2 and 3. The paired questions correlations noted in bold in Table 2 and the correlations matrix of 
Table 3 showed positive correlations amongst questions 1 and 2 as well as between questions 3 
and 4. Questions 1 and 2 have an r of .658 (p = .000) while questions 3 and 4 have an r of .575 

(p = .002). 


Table 2. Paired Questions Correlations 



N 

Correlation 

Sig. 

Pair 1 Q1 & 
_|Q2 

27 

.658 

.000 

Pair 2 Q3 & 

1 !Q4 

27 

.575 

.002 


The strong correlations are indicative that respondents were consistent in their answers to the two 
sets of questions, providing further evidence that the answers have validity and add to the 
perspective of a collaborative thinking process between the staff. 

Table 3. Correlations Matrix for Questions 1-4. 



Qi 

Q2 

Q3 

Q4 

Q1 Pearson Correlation 

l 

, 658 (**) 

-.004 

-.006 

Sig. (2-tailed) 


.000 

.985 

.974 

[n 

27 

27 

27 

27 

Q2 Pearson Correlation 

. 658 (**) 

1 

.011 

.109 

Sig. (2-tailed) 

.000 


.955 

.589 

1 |n 

27 

27 

27 

27 

Q3 Pearson Correlation 

-.004 

.011 

1 

. 575 (**) 

Sig. (2-tailed) 

.985 

.955 


.002 

1 |n 

27 

27 

27 

27 

Q4 Pearson Correlation 

-.006 

.109 

. 575 (**) 

1 

Sig. (2-tailed) 

.974 

.589 

.002 


[n 

27 

27 

27 

27 


** Correlation is significant at the 0.01 level (2-tailed). 

When the mean response for each question is calculated by professional type, as noted in Table 
4, (1 for special educators/2 for speech therapists), the results for 
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Questions #1-4 again show consistency. 
Table 4. Mean Scores by Professional Type 


Ques. 

# 

Prof. 

Type 

N 

Mean 

IqT 

1.00 

17 

4.1176 


2.00 

10 

3.9000 

Q2 

1.00 

17 

3.9412 


2.00 

10 

3.8000 

Q3 

1.00 

17 

3.8824 


2.00 

10 

4.3000 

Q4 

1.00 

17 

4.0000 


2.00 

10 

4.2000 

Q5 

1.00 

17 

3.5882 


2.00 

10 

4.4000 


The results for Question # 5 on the other hand indicated a notable difference between the 
opinions of the special educators and speech therapists. Question # 5 asked if the respondent 
believed push-in therapy was beneficial to the student. Interestingly, speech therapists responded 
with a more affirmative answer in comparison to the teachers, 4.40 vs. 3.58, respectively. As 
noted in Table 5 and indicated in bold, the difference represented a probability ratio of .050 and 
had statistical significance, indicating a 94% likelihood that this is a real and actual statistical 
difference in the scores between the two disciplines. The cause of this variation is speculative. 
Reasons could include the enhanced ability of the speech therapists to detect student progress in 
their relevant discipline to a biased desire by the speech therapists to see their students succeed. 

Table 5. Independent Sample T - Test for Question # 5 Showing Statistical Significance in 
Difference in Answers Between Speech Therapists and Special Educators. 


t-test for Equality of Means 




t 

df 

Sig. (2- 
tailed) 

Mean 

Difference 

Std. Error 
Difference 

95% Confidence 
Interval of the 
Difference 



Lower 

Upper 

Lower 

Upper 

Lower 

Upper Lower 

Q.5 

Equal variances 

-2.057 

25 

.050 

-.81176 

.39466 

-1.62459 .00106 


assumed 


Table 6 provided the statistical evidence for the above difference and further analysis of the 
results for Question # 5 when delineated according to professional discipline. This table gives the 
range of answers to Question # 5 and shows in bold that 50% of the speech therapists “strongly 
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agreed” (Response Code # 5.00) that their students benefited from push-in therapy in comparison 
to only 5.9% of the teachers. 


Table 6. Discipline Specific Perspectives on Push-in Speech Therapy 



Question # 5 

Total 

Range of 

Scores 

.No 

Selection 

2.00 

3.00 

4.00 

5.00 


Proftype 

1 [Frequency 

1 

1 

2 

12 

1 

17 


% within 
proftype 

5.9% 

5.9% 

11.8% 

70.6%] 

5.9% 

100.0°/J 


% of Total 

3.7% 

3.7% 

7.4% 

44.4%] 

3.7% 

63.0%| 


2 |Frequency 

0 

0 

1 

4 

5 

10 


% within 
Iproftype 

.0%] 

.0%] 

10.0%] 

40.0%] 

50.0% 

100.0°/J 


% of Total 

.0%] 

.0%] 

3.7% 

14.8%] 

18.5%] 

37.0%j 

Total 

Frequency 

1 

1 

3 

16 

6 

27 

% within 
jproftype 

3.7% 

3.7% 

11.1% 

59.3%] 

22.2%] 

100.0°/J 

% within Q5 

100.0% 

100.0%] 

100.0%] 

100.0%] 

100.0%] 

100.0% 

% of Total 

3.7% 

3.7% 

11.1% 

59.3%] 

22.2%] 

100.0%] 


In summary, the data indicated that the questions being addressed in this evaluation were 
answered in the affirmative. Specifically, quantitative analysis indicated the following: 1) the 
push-in/pull-out criteria were useful; 2) the process generated by use of the criteria facilitated 
collaboration; 3) collaboration was enhanced by the implementation of push-in therapy; 4) push- 
in was successfully incorporated into the classroom, and finally; 5) push-in therapy was 
beneficial to students. 


Comments 

Besides the statistical analysis conducted as part of this evaluation, 22 of the 27 respondents 
provided written comments. 14 special educators gave commentary while 8 speech therapists 
gave input. This translated into 60.9% and 66.7% of the respondents, respectively, with an 
aggregate percentage of 62.9%. An interpretation of the comments along with a generous sample 
of these comments is provided below. 

When it came to comments in response to Question # 5, (Was push-in therapy beneficial to your 
students), all 17 respondents provided specific feedback noting specific improvements in growth 
and development. Their comments ranged from statements that referred to the ability of the 
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student to better generalize ( 11 or 65 %), develop enhanced pragmatic, or social communication 
skills ( 8 or 47%), or receive added motivation and/or added attention ( 3, or 18%). In addition, 
18% of the special educators referred to the added learning that took place amongst the 
classroom staff when push-in therapy occurred, citing their ability to observe, and leam, speech 
therapists in action. Finally, the range of comments in relation to Question # 5 was greater for 
special educators, with 9 variations in comparison to only 2 for speech therapists. The possible 
reasons for this are varied, including the fact that there were more educators who provided 
comments in comparison to speech therapists. 

Below is a representative selection of comments made by respondents. The respondent’s 
discipline is indicated by the following initials. SpEd = special educator, SLP = speech language 
pathology. 

SpEd - “Those students that received push-in exhibited improvement in their expressive and 
pragmatic language.” 

SpEd - “The students showed an increase in their language skill level.” 

SpEd - “Push-in speech therapy is beneficial for students who require support in 
developing social language skills, play skills, etc. It gives students the opportunity to grow in 
their natural environment.” 

SpEd - “The student receives more attention and assistance therefore skills are able to come out 
that otherwise may not.” 

SLP - “Improvement generally noted informally - by increased ability to meet their IEP goals in 
the presence of classroom distractedness, showing a generalization of their skills.” 

SpEd - “The children who have had push-in lessons have showed increased social language, 
increased attention during speech activities, and it is fun and motivating.” 

SpEd - “There has been an increase in vocalization and sentence length in the language of many 
of the children! They are commenting on things that they observe as well as make requests.” 

SLP - “Push-in therapy provides carry-over/generalizations of specific goals targeted in 
individual speech therapy sessions. A child may do well 1:1, but it is important that they can 
utilize skills required in pull-out sessions in academic and social settings.” 

As with the quantitative data analysis, review of the qualitative data provides strong indication 
that both disciplines found the push-in process to be beneficial to students. 

In addition to the comments regarding push-in therapy, additional comments were provided by 
12 of the 17 commentary respondents, including 10 special educators and 2 speech therapists. 

The contents of additional comments ranged from general observations regarding their teams, ( 
25%), endorsements of the criteria, its use, and/or the overall push-in/pull-out evaluative process, 
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( 50%), criticisms of the tool, the process, its applicability or attempting to manipulate 
professional decision making (17%), to stating the need for additional training in how to provide 
push-in therapy (8%). As with comments in response to Question # 5, there was much more 
variation in additional responses amongst the special educators in comparison to the speech 
therapists. 

Below are some sample additional comments made by special educators and speech therapists. 
SpEd - “I think the therapists need more training on push-in activities and preparation” 

SLP - “I do not think this tool was useful. For experienced teachers and therapists, we don’t need 
a tool to determine if a child needs push-in/pull-out therapy. Possibly useful for new, 
inexperienced staff’ 

SpEd - “The assessment process helped the team understand and come to a unified conclusion 
on how a child should receive his/her therapy.” 

SLP - “I feel many people have different views on the push-in/pull-out model as well as the 
ways it can be implemented. Training should be on-going. Expectations should be clear. There 
should be accountability and follow through. 

SpEd - “If it helps the therapists then it is a good tool. We kind of already had all of these 
concepts in place. It is a bit superfluous.” 

SpEd - Push-in therapy seems to be an effective tool towards helping the child leam in a more 
natural teaching environment. Where 1:1 pull-out therapy is necessary for most children it is 
helpful to have a tool such as this to determine when a child would be ready to benefit from 
push-in therapy. 


Conclusions and Implications 

Through the use of a five item, Likert scale survey, this evaluation provided feedback on the use 
of criteria utilized to determine if a student should be provided speech therapy on a push-in basis 
in the contextual setting of the student’s classroom. Additional information was retrieved and 
offered insight into the collaborative engagements that resulted from use of the criteria and the 
resulting push-in therapy. Finally, the survey attempted to discern through the participants of the 
study whether their students benefited from push-in speech language therapy. 

Based on the literature, the author developed a hypothesis that use of a push-in/ pull-out criteria 
would assist in detennining which students would benefit from push-in, or integrated therapy as 
well as the other assumptions noted above. The objective of this evaluation was to test these 
hypotheses, and as noted above, the results provided confirmation, thus demonstrating the 
criteria’s facilitation of push-in therapy, interdisciplinary collaboration and enhanced student 
development. Furthermore, the results showed significant consistency and when correlated with 
the comments from both provider groups, the outcomes strongly favored the perspective that use 
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of the push-in/pull-out criteria was successful in assisting the integration of speech therapy into 
the classroom setting; expanding the collaborative engagements amongst teachers and speech 
therapists; and fostering greater communicative skill in students. 

Limitations of Survey and Evaluation 

Critics could argue that attributing the enhanced collaborative engagements and the success of 
push-in therapy to use of the criteria is inconclusive. The reasons for this are varied and include 
the following potential oversights and errors that can occur in determining the program effect, its 
impact assessment and long term outcome monitoring; 

- Gathering and analysis of additional data relevant to the development of students’ 
communicative and language skills. The use of experimental research methodology, including 
the incorporation of a control group, and/or the analysis of other variables in the existing 
nonexperimental study, would provide more robust and powerful data in support of push-in 
therapy. In the current study, any inferences of student progress had to be determined without the 
benefit of pre-testing or other comparative statistical benchmarks that occur in an experimental 
study when a control group exists. This lends itself to questions such as how much did the 
integrated push-in therapy program produce positive outcomes above what would be expected 
from the provision of isolated, pull-out therapy? The detennination that program effects were 
arrived at via use of the criteria and provision of push-in therapy could have been inferred, at 
least by a few of the participants. The identified improvements could have been arrived via other 
interventions and/or unintended variables. And while a positive effect was reported without any 
obvious bias or preponderance for one type of student in comparison to another, many would 
state that the results would be more valuable if there was an accounting of moderator variables 
such as class size, staffing ratio, inclusionary vs. self-contained classes. 

In addition to moderator variables, what effect, if any, did mediator variables have on the 
findings? Is it possible that initial student success after the initiation of push-in therapy led to 
further adoption of the push-in model? If yes, this lack of clarity could lead to undue emphasis 
on variables of lesser importance in the proximal and distal outcome and impact planning. 

The “corruptibility of indicators” (Rossi, et. ah, 2004 p. 227) could lead to the enhancement 
of the results by the participating teachers and therapists in order to give favorable responses on 
the survey tool in order to please the researcher. 

The existence of spurious relationships and intervening direct and indirect variable(s) could 
have impacted the results (Johnson & Christensen, 2008). Could answers to questions one and 
two have an effect on how you answer question three, four and five? Could the completion of the 
survey questions influence the person’s perspectives on collaboration and produce a causal 
outcome whereby the subject has a more favorable view of push-in therapy? 

- Questioning whether collaboration and/or push-in therapy could have occurred without use of 
the push-in/pull-out criteria would have been a useful question to ask. In addition, the 
determination of successful push-in therapy is based on somewhat subjective interpretation by 
the respondents and therefore less reliable than objectively answered questions. 
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Considering the limited sample size and pre-existence of a collaborative environment at the 
preschool, is difficult to generalize the study’s findings to other locations. 

What effect, if any, did the interpersonal relationship of the teachers and speech therapists 
have on the success of the criteria’s use, the implementation of push-in as well as the 
collaborative engagements amongst the individuals involved? 

Besides the above noted limitations, one other shortcoming is the lack of literature regarding 
how one should go about creating an integrated therapy program or simply the use of criteria in 
determining the provision of push-in versus pull-out therapy. While an abundance of literature 
was reviewed and cited, most of it espoused the benefits of collaboration and/or integrated 
therapy. This shortcoming did not seem to negatively impact on the outcomes but additional 
sources could have nevertheless added to the literature used to support the program. 

Further Research 

With these criticisms in mind, further research and inquiry into integrated therapy and its 
associated processes is warranted. Besides additional inquiry into the processes associated with 
speech language pathology, other therapeutic interventions are worthy of investigation, including 
occupational therapy, physical therapy, counseling and play therapy. An evaluation applicable to 
all disciplines would include mixed research into the criteria and processes used to detennine 
how therapy is to be provided as well as the relationships between the type of therapy being 
provided and the ease at which integrated therapy is implemented. One would also need to 
consider the role collaborative culture and discipline affiliation has within an organization and 
the impact it has in the implementation of additional collaborative measures. Even further insight 
would be achieved by determining the roles experience and interpersonal maturity has in 
facilitating the push-in process. 

Finally, future investigation should also detennine if the collaborative processes, including push- 
in/integrated therapy, has any real benefits for children. This avenue of research should collect 
information on both push-in and pull-out therapy and its impact on children, in both a cross- 
sectional as well as longitudinal manner, providing the field with insight into the immediate as 
well as long term implications of these therapeutic modalities. 

Dissemination of Results 

Both primary and secondary dissemination of results occurred. Primary dissemination was to 
program administration with secondary dissemination to the teachers, therapists, parents and 
other interested parties. 


Utilization of Evaluation Results 

The evaluation results were used in the following manner: 
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• Reinforced to administration and staff the benefits (including enhanced student 
development) of the push-in/pull-out criteria, collaboration and push-in/integrated 
therapy. 

• As per commentary recommendation, the push-in/pull-out criteria and tool should be 
modified in order to provide more explicit direction to staff, help enhance future usage 
and facilitate greater effectiveness of process. 

• Aid in the facilitation of enhanced collaboration and push-in therapy through the 
incorporation and utilization of the push-in/pull-out criteria into other disciplinary 
services such as Occupational Therapy. 

Finally, the results can be used to assist in the development of further evaluative studies designed 
to assess other aspects of a collaborative culture within a special education environment. 

Conclusion 

Based on findings from this study, the utilization of criteria designed to assist service providers 
determine the appropriateness of push-in/integrated and/or pull-out therapy was found to be 
successful in its implementation at the preschool in question. 

The success of this study, whether it was the ability to generate the data necessary to conduct this 
evaluation, the confirmation of the value of criteria, collaboration and integrated therapy, has 
affirmed several assumptions the management team had regarding collaboration and all of its 
characteristics, qualities and variations. First, as Cain (2012) explains, collaboration presents a 
conundrum of by pointing out what she calls the contradictory impulses of human nature: the 
need for companionship while also needing privacy and autonomy. Second, without 
administration spearheading the process, the engagement of professionals in collaborative 
endeavors will not occur to any substantial extent. Third, the institutionalization of collaboration, 
through such means as established organizational philosophy, regularly scheduled team 
meetings, team focused membership and the like, is necessary for collaboration to occur but not 
a guarantee that it will succeed. Finally, people are not bom collaborators and therefore need 
training, guidance and support in order to be successful. The comments gathered as part of the 
study support these beliefs and add impetus to efforts to refine the existing processes and 
improve on the ability of staff to work together and to maximize the growth and development of 
all students. 
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Attachment A 

Assessment Instrument for Determining Push-In Speech-Language Therapy 

Applicability of any of the following criteria to the student in question identifies that child as a 

candidate for push-in speech therapy. 

STUDENT NAME:_ 

_The child is receptive to therapy within the classroom. 

_The child needs repetition of a learned skill within a familiar environment. 

_The child has mastered a particular skill within the treatment session and is in 

need of mastering generalization of that skill in another environment. 

_The child has difficulty transitioning. 

_The child has an adequate attention span in order to focus on his/her individual goals 

in a distractible environment. 

_The classroom teacher has concerns about a developmental skill that has not been 

brought to the attention of the therapist via the I.E.P. 

_Session goals identified by the team. These may include but are not limited to: 

promoting social language 
modeling the use of visuals 
modeling communicative temptations 
identifying strategies to improve attention 
modeling sequencing skills 
developing symbolic play 
developing spontaneous language 
promoting generalization of language skills 

_Optimal classroom times for push in therapy identified by the team. These may 

include but are not limited to: 
meal/snack time 
circle time 
cooking lessons 
arts and crafts 
story time 
free play 
computer 

Comments or Additional Observations: 
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Attachment B 


Assessment Instrument for Determining Pull-Out Speech-Language Therapy 

Applicability of any of the following criteria to the student in question identifies that child as a 
candidate for pull-out speech therapy. 


STUDENT NAME: 


The child needs a small, quiet environment in which to learn or improve a skill. 
The child needs complete 1:1 attention to leam a skill. 

The child needs repeated practice of a particular skill in order to improve 
performance. 

The child needs to use special equipment/materials or more space than is 
available in the classroom. 

The child is more motivated to comply and attend to challenging activities by a 
pull-out session due to the individualized nature of this type of session. 

Pull-out sessions will benefit the child’s self esteem. 

Pull-out sessions are necessary when the current classroom activity does not 
address the child’s particular I.E.P. goal(s). 


Comments or Additional Observations: 
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Attachment C 

Push-in/Pull-out Criteria Assessment Survey 


Instructions: Indicate on the scales below your perceptions of the Push-in/Pull-out Criteria 
Assessment Instruments (attached) and the collaborative processes that ensued. 

Circle the answer you are most in agreement with. 

1. The Push-in/Pull-out Criteria Assessment Instrument was useful in determining which 
students should be seen for Speech Therapy on a push-in or pull-out basis. 

Strongly Disagree Not Sure Agree Strongly 
Disagree Agree 


2. The Push-in/Pull-out Assessment Instrument enhanced collaboration at the time the team 
was determining if students should be seen on a push-in or pull-out basis. 


Strongly Disagree Not Sure Agree Strongly 

Disagree Agree 


3. Collaboration between you and the teacher or speech therapist was enhanced as a result of 
the provision of push-in speech therapy in the classroom(s) you work in? 


Strongly Disagree Not Sure Agree Strongly 

Disagree Agree 


4 . Push-in therapy was successfully incorporated into the classroom setting? 


Strongly Disagree Not Sure Agree Strongly 

Disagree Agree 


5. Push-in speech therapy was beneficial to those students who received it. 

Strongly Disagree Not Sure Agree Strongly 
Disagree Agree 
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If your response to Question # 5 is Agree or Strongly Agree, please indicate how you came to 
this conclusion and note any quantifiable measures that note improvement in the student’s (or 
students’) developmental levels. 


Finally, please add any comments you would like to make about the Push-in/Pull-out Assessment 
Tool, its implementation and/or push-in versus pull-out 

therapy._ 
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Attachment D 


To: All Classroom Teams 
From: Senior Speech Pathologist 

RE: Speech Therapy Criteria for Push-in/Push-out Therapy 


Please find attached the criteria for Speech Push-in/Pull-out therapy. After collaborating as a 
team, please complete the checklist for each student. There is no minimum requirement of 
criteria that render a child “eligible” for either form of therapy. The checklist is designed to be a 
guide for determining which students will benefit from push-in/pull-out therapy as well as 
identifying the therapeutic goals being targeted for each session. Please contact me if you have 
any questions regarding this checklist. 

Thanks! 
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